Image processing method and device for performing mosquito noise reduction

ABSTRACT

The present invention provides an image processing method for processing an image. The method includes: detecting at least an edge in the image; determining at least a pixel window including the edge; detecting whether a mosquito noise exists in the pixel window; and filtering out the detected mosquito noise in the pixel window.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing method and device,and more particularly, to an image processing method and device forfiltering out the detected mosquito noise(s) near an edge in an image.

2. Description of the Prior Art

In the technical field of image processing, MPEG (Moving Picture ExpertsGroup) and/or JPEG (Joint Photographic Experts Group) compression iscommonly utilized in applications that encode, decode, transmit, store,or play images. Typically, DCT (Discrete Cosine Transform) is performedon the images during compression, thereby causing some problems. Forexample, some randomly distributed dotted noises may occur in the imagesduring compression due to that the DCT algorithm typically discards highfrequency portions of images when processing images. Those noises causedby such a destructive compression algorithm are called mosquito noises.

Conventionally, mosquito noises in the image can be removed by the aidof related compression/decompression information generated duringMPEG/JPEG compression. However, due to variety of image media presentlyused in the market, it is often hard to know the origin of the image,and it is also hard for a back end device processing the image to getthe related compression/decompression information. Thus, how toprecisely remove mosquito noises from the image under such condition iscertainly a tough issue.

SUMMARY OF THE INVENTION

It is therefore one of the objectives of the present invention toprovide an image processing method and device for detecting a mosquitonoise near at least an edge in an image and filtering out the detectedmosquito noise to solve the above mentioned problems.

The present invention discloses an image processing method forprocessing an image, comprising: detecting at least an edge in theimage; determining at least a pixel window including the edge; detectingwhether a mosquito noise exists in the pixel window; and filtering outthe detected mosquito noise in the pixel window when the mosquito noiseexists in the pixel window.

The present invention also discloses an image processing device forprocessing an image, comprising: an edge detector, for detecting atleast an edge in the image; a mosquito detector, coupled to the edgedetector and comprising: a determining unit, for determining at least apixel window according to the edge; and a detecting unit, for detectingwhether a mosquito noise exists in the pixel window; and a mosquitofilter, coupled to the edge detector and the mosquito detector, forfiltering out the detected mosquito noise when the mosquito noise existsin the pixel window.

These and other objectives of the present invention will no doubt becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of the preferred embodiment that isillustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram illustrating an embodiment of animage processing device of the present invention.

FIG. 2 is a flowchart illustrating an embodiment of an image processingmethod of the present invention.

FIG. 3 is a continued flowchart corresponding to FIG. 2.

FIG. 4 is a schematic diagram illustrating a pixel window and aplurality of preset patterns.

DETAILED DESCRIPTION

Please refer to FIG. 1. FIG. 1 is a schematic block diagram illustratingan preferred embodiment of an image processing device 101 of the presentinvention. The image processing device 101 is utilized for processingimages. In an embodiment of the present invention, the images aredigital images comprising a plurality of pixels, and pixel valuescorresponding to the pixels can be R, G, or B pixel values in a RGBformat, brightness or color values in a YUV format, or any other valuesindicating displaying characteristics of the pixels in other commonvideo formats. As shown in FIG. 1, the image processing device 101comprises an edge detector 102, a mosquito detector 103, a mosquitofilter 104, an edge enhancer 105, a counter 106, and a controller 107.The mosquito detector 103 is coupled to the edge detector 102 fordetecting a mosquito noise. The mosquito filter 104 is coupled to theedge detector 102 and the mosquito detector 103 for filtering out thedetected mosquito noise. The edge enhancer 105 is coupled to the edgedetector 102, the mosquito detector 103, and the mosquito filter 104 forperforming an edge enhancing process on a corresponding pixel windowafter the mosquito filter has filtered out the detected mosquito noise.The counter 106 is coupled to the mosquito detector 103 for counting atotal number of the detected mosquito noise(s) in an accumulativemanner. The controller 107 is coupled to the mosquito filter 104, theedge enhancer 105, and the counter 106 for controlling the mosquitofilter 104 according to the total number. Additionally, in thisembodiment, the mosquito detector 103 comprises a determining unit 201and a detecting unit 202. The determining unit 201 and the detectingunit 202 are coupled to each other. The determining unit 201 determinesor creating at least a pixel window including a detected edge.Additionally, according to an embodiment of the present invention, thedetecting unit 202 compares the pixel window with at least a targetpattern to determine whether the pixel window conforms to the targetpattern, thereby deciding whether a mosquito noise exists in the pixelwindow. The above comparison is not meant to be a limitation of thepresent invention, however. In other embodiments, the detecting unit 202can also utilize other detection method.

Please refer to both FIG. 2 and FIG. 3. FIG. 2 is a flowchartillustrating an embodiment of an image processing method of the presentinvention. FIG. 3 is a continued flowchart corresponding to FIG. 2. Asshown in drawings, the image processing method utilized by the imageprocessing device 101 in FIG. 1 comprises the following steps:

STEP 502: Detect at least an edge in an image;

STEP 504: Select a pixel window from a plurality of pixel windowscorresponding to the edge to be processed;

STEP 506: Compare the pixel window with at least a target pattern todetermine whether the pixel window conforms to the target pattern? Ifso, proceed to STEP 508; if not, jump to STEP 516;

STEP 508: Decide that a mosquito noise exists in the pixel window;

STEP 510: Add an increment (e.g. 1) to a total number of the detectedmosquito noise(s) so far;

STEP 512: Perform a low pass filtering process on the pixel window tofilter out the mosquito noise;

STEP 514: Perform an edge enhancing process on the pixel window;

STEP 516: Determine whether all pixel windows corresponding to the edgehave been processed? If so, proceed to STEP 520; if not, proceed to STEP518;

STEP 518: Select the next pixel window from the pixel windowscorresponding to the edge, and then return to STEP 506 for processingthe next pixel window;

STEP 520: Determine whether a total number of all mosquito noisesaccumulated so far is less than a first threshold value? If so, proceedto STEP 522; if not, proceed to STEP 524;

STEP 522: Skip STEPs 512 and 514 when processing the next edge;

STEP 524: Perform STEPs 512 and 514 when processing the next edge;

STEP 526: Determine whether a total number of all mosquito noisesaccumulated so far is greater than a second threshold value? If so,proceed to STEP 528; if not, jump to STEP 530;

STEP 528: Increase the size of a current pixel window utilized forfiltering out the mosquito noise;

STEP 530: Determine whether all edges in the image have been processed?If so, end the process herein; if not, proceed to STEP 532; and

STEP 532: Reset the total number of all mosquito noises to be an initialvalue (e.g. 0), and then return to STEP 504 for processing the next edgein the image.

The details as to how the image processing device 101 performs the imageprocessing method as shown in FIG. 2 and FIG. 3 are described asfollows. Please note that, edges of real objects in an image processedby a discrete cosine transform (DCT) algorithm often become unclear dueto that the DCT algorithm typically discards high frequency portions ofimages when processing images. For human eyes, some unwanted irregulardots (noises) thus occur near the edges. Roughly speaking, mosquitonoises are those unwanted irregular dots occurring near the edges of thereal objects in the image. Thus, the first step of the image processingmethod of the present invention is to find existing edges in an image.Please refer to FIG. 2. First, the edge detector 102 detects at least anedge in an image (STEP 502). Sobel filters and Laplace filters are twocommon filters for detecting edges. Since Sobel filters and Laplacefilters are well known to those skilled in the art, the relateddescription is omitted herein for simplicity. Next, the determining unit201 of the mosquito detector 103 selects a pixel window from a pluralityof pixel windows corresponding to the edge to be processed (STEP 504).Additionally, the pixel windows are formed along the edge. Next, thedetecting unit 202 of the mosquito detector 103 compares the pixelwindow with at least a target pattern to determine whether the pixelwindow conforms to the target pattern (STEP 506).

Please refer to FIG. 4. FIG. 4 is a schematic diagram illustrating apixel window 301 and a plurality of preset patterns 401, 402, 403. Asshown in FIG. 4, the determining unit 201 determines the pixel window301 by combining a central pixel point E corresponding to the edgedetected by the edge detector 102 and 8 surrounding pixel points A, B,C, D, F, G, H, I. Additionally, as described above, pixel valuescorresponding to the 9 pixel points can be R, G, or B pixel values in aRGB format, brightness or color values in a YUV format, or any othervalues indicating displaying characteristics of the pixels in othercommon video formats. The preset pattern 402 out of the preset patterns401, 402, 403 as shown in FIG. 4 is the target pattern of the presentinvention. Directions of edges in the preset pattern 402 are veryirregular, implying that mosquito noises may exist therein. This is onlyone of the examples of the present invention, however. In otherembodiments, the target pattern can also be defined according to otherprinciples for detecting mosquito noises. For example, in otherembodiments, it is decided that mosquito noises exist when the pixelwindow corresponds to any one of the preset patterns 401, 402, 403. Suchan example also falls in the scope of the present invention. As shown inFIG. 4, if the pixel window 301 satisfies a characteristic equation of“A=B=C=D=E=F && G=H=I && A!=G”, the pixel window 301 conforms to thepreset pattern 401. If the pixel window 301 satisfies a characteristicequation of “A=B=C=E=G=H=I && D=F && A!=D”, the pixel window 301conforms to the target pattern (i.e. the preset pattern 402). If thepixel window 301 satisfies a characteristic equation of “A=D=G=B=E=H &&C=F=I && A!=C”, the pixel window 301 conforms to the preset pattern 403.

Please note that although the embodiment is exemplified using a 3×3pixel window having 9 pixels, this is not meant to be a limitation ofthe present invention. Any other pixel window of other sizes, such as5×5 and 7×7, falls in the scope of the present invention. Additionally,the preset patterns 401, 402, 403 in FIG. 4 are merely one example andare not meant to be a limitation of the present invention. Other presetpatterns having different styles can also be utilized to define thetarget pattern provided that the principles of the present invention aremet.

Please return to FIG. 2. If it is determined in STEP 506 that the pixelwindow 301 conforms to the target pattern (i.e. the preset pattern 402),the mosquito detector 103 decides that a mosquito noise exists in thepixel window 301 (STEP 508). In other words, a mosquito noise existsnear the edge. Additionally, in another embodiment of the presentinvention, STEP 506 can also be achieved by determining whether thenumber of major edges that occur most often in a pixel window (having asize of 3×3, 5×5, or 7×7) is greater than half the total number of alledges occurring in the pixel window. If that condition is notestablished, it means that the edges of the pixel window do not appear aspecific directional trend. For example, that condition is notestablished when directions of the edges are very irregular as shown inthe target pattern (i.e. the preset pattern 402) in FIG. 4 of thepresent invention. In such cases, it is determined that a mosquito noiseexists in the pixel window. Please note that, in other embodiments,mosquito noises can also be detected by other methods. For example, apixel difference filter can also be utilized to determine mosquitonoises. Next, the counter 106 adds 1 to a total number of the detectedmosquito noise(s) so far (STEP 510). The mosquito filter 104 performs alow pass filtering process on the pixel window 301 to filter out themosquito noise (STEP 512). Typically, a median filtering process isutilized as the low pass filtering process herein for filtering out highfrequency noises with a satisfying result. Additionally, because thecomputation for filtering out the mosquito noise makes the edge unclear,the edge enhancer 105 will perform an edge enhancing process on thepixel window 301 (STEP 514). Typically, 3×3 Gaussian smoothing filter isutilized as an edge enhancer herein. Please note that, in thisembodiment, STEP 512 can be performed either before or after STEP 514,and the same processing result of the present invention can be achievedin either order.

Contrarily, if it is determined in STEP 506 that the pixel window 301does not conform to the target pattern (i.e. the preset pattern 402),the process jumps to STEP 516. Then, the mosquito detector 103determines whether all pixel windows corresponding to the edge have beenprocessed (STEP 516).

If it is determined in STEP 516 that the image processing device 101 hasnot processed all pixel windows, the determining unit 201 of themosquito detector 103 will select the next pixel window not yetprocessed from the pixel windows corresponding to the edge (STEP 518).The process then returns to STEP 506 for processing the next pixelwindow. Additionally, in this embodiment, the image processing methodutilizes a pixel window as a basic unit to process an image. In otherwords, after one pixel window corresponding to an edge in the images hasbeen processed by the above steps (i.e. from STEP 506 to STEP 514), thenext pixel window will be processed. Such a procedure will go on untilall pixel windows corresponding to the edge in the image have beenprocessed. Please note that the pixel windows are not necessary to beprocessed in the order as disclosed above.

Contrarily, if it is determined in STEP 516 that all pixel windowscorresponding to the edge have been processed, the controller 107 willdetermine whether a total number of all mosquito noises accumulated sofar is less than a first threshold value (STEP 520). If the total numberof the mosquito noises so far is less than the first threshold, it meansthat the detected “mosquito noises” are not real mosquito noises. Inother words, the detected “mosquito noises” are just usual noises. Insuch a case, the controller 107 disables the mosquito filter 104 and theedge enhancer 105 temporarily. Therefore, when the image processingdevice 101 processes the next edge, STEPs 512 and 514 will be skipped(STEP 522), thereby avoiding image quality deterioration due tounnecessary mosquito noise filtering processes. Please note that thoseskilled in the art can appropriately design the first threshold valueaccording to the practical requirements (or experimental results) afterunderstanding the principles of the present invention as disclosedabove. Contrarily, if the total number of the mosquito noises so far isnot less than the first threshold value, the controller 107 will enablethe mosquito filter 104 and the edge enhancer 105. Thus, when the imageprocessing device 101 processes the next edge, STEPs 512 and 514 will beperformed (STEP 524).

Next, the controller 107 determines whether a total number of allmosquito noises accumulated so far is greater than a second thresholdvalue (STEP 526). If the total number of the mosquito noises so far isgreater than the second threshold value, the controller 107 willincreases the size of a current pixel window utilized for filtering outthe mosquito noise (STEP 528). For example, in this embodiment, the sizeof the pixel window 301 can be increased from original 3×3 to 11×11 orother values in STEP 528. Please note that those skilled in the art canappropriately design the second threshold value according to thepractical requirements (or experimental results) after understanding theprinciples of the present invention as disclosed above. Contrarily, ifthe total number of the mosquito noises so far is not greater than thesecond threshold value, the process will jump to STEP 530. Then, themosquito detector 103 will determine whether all edges in the image havebeen processed (STEP 530). If it is determined that all edges in theimage have been processed, the whole process will end herein.Contrarily, if it is determined that the image processing device 101 hasnot processed all edges in the image, the counter 106 will reset thetotal number of all mosquito noises to be 0 (STEP 532), and then theprocess will return to STEP 504 for processing the next edge in theimage.

In this embodiment, for each edge, the controller 107 does not determinewhether the total number of all mosquito noises accumulated so far isless than the first threshold value or greater than the second thresholdvalue unless all pixel windows corresponding to the edge have beenprocessed. This is not meant to be a limitation of the presentinvention, however. Furthermore, the controller 107 can also determinewhether the total number of all mosquito noises accumulated so far isless than the first threshold value or greater than the second thresholdvalue at any other time point during the whole process. For each edge,the controller 107 can determine whether the total number of allmosquito noises accumulated so far is less than the first thresholdvalue or greater than the second threshold value after at least 10 pixelwindows have been processed, for example.

The image processing method and device of the present invention can beutilized for filtering out mosquito noises, thereby upgrading imageclarity and effectively sharpening edges of real objects in an image.Additionally, the method and device does not needcompression/decompression information generated during MPEG (MovingPicture Experts Group) or JPEG (Joint Photographic Experts Group)compression, original images before compression, or any other relatedinformation on a temporal basis. In other words, the present inventioncan be applied to any image for detecting and filtering out mosquitonoises whether the image is compressed or not, whether the image isenlarged (e.g. from 8×8 blocks to 16×16 blocks) or not, whether theimage is shrunk (e.g. from 16×16 blocks to 8×8 blocks) or not, andwhether the image is shifted or not.

Please note that, the techniques and principles of the present inventionas disclosed in the above embodiment(s) can be applied to various imageprocessing devices including still picture processing devices, such asdigital cameras, and motion video systems, such as LCDs (Liquid CrystalDisplays), LCD TVs, and digital TVs. Those skilled in the art can easilyapply the present invention to other related technical fields afterunderstanding the techniques and principles of the present invention asdisclosed in the above embodiment(s).

Additionally, those skilled in electronic circuit design, digital signalprocessing, or digital image processing, can also utilize any feasibleprinciples as to hardware circuit design or software programming toaccomplish the image processing method and device of the presentinvention after understanding the techniques and principles of thepresent invention as disclosed in the above embodiment(s).

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the invention.

1. An image processing method for processing an image, comprising:detecting at least an edge in the image; creating at least a pixelwindow including the edge; detecting whether a mosquito noise exists inthe pixel window; and filtering out the detected mosquito noise in thepixel window when the mosquito noise exists in the pixel window.
 2. Themethod of claim 1, wherein the step of detecting whether the mosquitonoise exists in the pixel window comprises: comparing the pixel windowwith at least a target pattern so as to determine whether the pixelwindow conforms to the target pattern; and deciding that the mosquitonoise exists in the pixel window if the pixel window conforms to thetarget pattern.
 3. The method of claim 1, wherein the step of filteringout the detected mosquito noise in the pixel window comprises:performing a low pass filtering process on the pixel window to filterout the detected mosquito noise.
 4. The method of claim 1, furthercomprising: performing an edge enhancing process on the pixel window. 5.The method of claim 2, further comprising: performing an edge enhancingprocess on the pixel window.
 6. The method of claim 5, wherein the stepof filtering out the detected mosquito noise in the pixel windowcomprises: performing a low pass filtering process on the pixel windowto filter out the detected mosquito noise after the edge enhancingprocess has been performed on the pixel window.
 7. The method of claim2, further comprising: counting a total number of the detected mosquitonoise(s) in an accumulative manner; and controlling the step offiltering out the detected mosquito noise in the pixel window accordingto the total number.
 8. The method of claim 7, wherein the step ofcontrolling the step of filtering out the detected mosquito noise in thepixel window according to the total number comprises: pausing the stepof filtering out the detected mosquito noise in the pixel window whenthe total number is less than a first threshold value.
 9. The method ofclaim 7, wherein the step of controlling the step of filtering out thedetected mosquito noise in the pixel window according to the totalnumber comprises: increasing the size of a current pixel window when thetotal number is greater than a second threshold value.
 10. The method ofclaim 1, further comprising: counting a total number of the detectedmosquito noise(s) in an accumulative manner; and controlling the step offiltering out the detected mosquito noise in the pixel window accordingto the total number.
 11. The method of claim 10, wherein the step ofcontrolling the step of filtering out the detected mosquito noise in thepixel window according to the total number comprises: pausing the stepof filtering out the detected mosquito noise in the pixel window whenthe total number is less than a first threshold value.
 12. The method ofclaim 10, wherein the step of controlling the step of filtering out thedetected mosquito noise in the pixel window according to the totalnumber comprises: increasing the size of a current pixel window utilizedfor filtering out the detected mosquito noise in the pixel window whenthe total number is greater than a second threshold value.
 13. An imageprocessing device for processing an image, comprising: an edge detector,for detecting at least an edge in the image; a mosquito detector,coupled to the edge detector and comprising: a determining unit, fordetermining at least a pixel window including the edge; and a detectingunit, for detecting whether a mosquito noise exists in the pixel window;and a mosquito filter, coupled to the edge detector and the mosquitodetector, for filtering out the detected mosquito noise when themosquito noise exists in the pixel window.
 14. The device of claim 13,wherein the detecting unit compares the pixel window with at least atarget pattern to determine whether the pixel window conforms to thetarget pattern, and wherein the mosquito detector decides that themosquito noise exists in the pixel window if the pixel window conformsto the target pattern.
 15. The device of claim 13, wherein the mosquitofilter performs a low pass filtering process on the pixel window tofilter out the detected mosquito noise when the mosquito detectordecides that the mosquito noise exists in the pixel window.
 16. Thedevice of claim 13, further comprising: an edge enhancer, coupled to theedge detector and the mosquito filter, for performing an edge enhancingprocess on the pixel window after the mosquito filter has filtered outthe detected mosquito noise.
 17. The device of claim 13, furthercomprising: a counter, coupled to the mosquito detector, for counting atotal number of the detected mosquito noise(s) in an accumulativemanner; and a controller, coupled to the counter and the mosquitofilter, for controlling the mosquito filter according to the totalnumber.
 18. The device of claim 17, wherein the controller disables themosquito filter when the total number is less than a first thresholdvalue.
 19. The device of claim 17, wherein the controller increases thesize of a current pixel window when the total number is greater than asecond threshold value.
 20. The device of claim 17, wherein the edgedetector is a Sobel detector.